NOTE ON THE GROUP OF AUTOMORPHISMS OF THE 

SPECTRAL BALL 
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Abstract. It is shown that the conjecture on the description of the 
group of automorphism of the spectral ball posed by Ransford and White 
C^I , is false. 

> 

o . 

| 7 | ■ Let f2 denote the spectral ball, that is a domain composed of n x n 

q ■ complex matrices whose spectral radius is less then 1. The natural question 

that arises in the study of the geometry of Q is to classify its group of 
automorphisms. It is seen that among them there are the following three 
■ forms: 

(i) Transposition: T : x i— > x t , 
^ ■ (ii) Mobius maps: M. aa : x >->■ 7(2; — a) (I — ax)' 1 , where a lies in the unit 

disc and |7| = 1, 

(hi) Conjugations: J u : x u(x)~ 1 xu(x), where u : ft — > M~ x is a conju- 
gate invariant holomorphic map, i.e. u(q _1 xq) = u(x) for each x G f2 
and q G M~ l . 

IO ! Ransford and White have asked in [3] whether the compositions of the 

three above forms generate the whole Aut(fi) - the group of automorphisms 
(Sj ■ of the spectral ball. The problems related to this question were considered 

among others in [I], [2], jl] and [5]. 

The aim of this short note is to show that the answer to the conjecture 
posed by Ransford and White is negative. More precisely, we will present 
an example of an automorphism of the spectral ball which is not generated 
by mappings (i), (ii) and (hi). 

To simplify the notation we focus our attention on the case n = 2. The 
similar examples working for n > 2 are self-evident. 

For the convenience of the reader we start with recalling some basic 
properties of the mappings (i), (ii) and (iii) which will be useful in the 
sequel. Let / be an automorphism of the spectral ball generated by (i), (ii) 
and (iii) and preserving the origin. Observe that J u °M. a ^ = M. a ^oJ uoM 
for any u : Q — > Ai' 1 , |a| < 1 and |7| = 1. Therefore, the automorphism 
/ is generated only by the mappings of the form (i) and (iii). Moreover, 
f'(0) is a linear automorphism of the spectral ball and it follows from [3] 
that f'(0).x = •ymxrn' 1 , x G f2, or f'(0).x = •ymx t m~ 1 , x G ft, for some 
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unimodular constant 7 and invertible matrix m. Note also that if the first 
possibility holds, i.e. f'(0).x = jmxm' 1 , then 7" 1 / is of the form (iii). 

Theorem 1 (Example 1). Let <p be an entire holomorphic function on C. 
Put 

fit \\ ( x n e-^ x ^x 12 \ 

Then f is an automorphism offl which is not generated by mappings of the 
form (i), (ii) and (iii), providing that ip is non- constant. 

Proof. It is clear that / preserves the spectrum, so /(fi) C Q. Thus, it 
suffices to observe that the inverse to / is given by the formula: 

What remains to do is to show that / is not generated by the mappings 
of the form (i), (ii) and (iii). 

It is clear that f'(0).x = rnxm" 1 , x G Cl, for some diagonal m G A^^" 1 , 
so we aim at proving that / is not a conjugation J v with v depending only 
on the conjugacy class of x. 

It is enough to observe that / restricted to the fiber & := {x G Q : 
detx = 0, trx = 1/2} is not of the form f(x) = nxn" 1 , x G Q, for any 
invertible n. To do it compute 

A A \\ _ f A e -^(A(l/2-A)) A 

1/2 -A 1/2- A J J ~ I e ^(A(i/2-A))( 1 / 2 _ A ) 1/2 -A, 



/ 



A G C, and use the trivial fact that for any a, (5 G C the mappings 
e v(A(i/2-A)) A an j a \ + (3, \ E C are not identically equal, provided that 
ip is non-constant (compare for example the singularity at 00). □ 

Remark 1. One can show that the conjugation 

(1) J[x) = diag(a(x), a -1 (;c))a; diag(a(x), a _1 (x)) _1 , 

where a is a non- vanishing holomorphic function on Q, is an automorphism 
of the spectral ball if and only if a depends only on Xn,x 22 and Xi 2 x 2 i. 

This remark describes more precisely the situation occurring in the Ex- 
ample [TJ Actually, the automorphism appearing in Example [1] is of the form 
([Tj) with a depending only on a;i 2 x 2 i (more precisely, a(x) = e~ Lp( - Xl2X21 " ) ) . 

Example 2. Put G fi, where a is 



x 2 \ x 22 J J ' \ a(xi 2 ) 1 r 
an entire holomorphic function. Then the conjugation f(x) = u(x)xu(x)^ 1 , 
x G f2, is an automorphism of Q non- generated by the automorphisms of 
the form (i), (ii) and (iii). 

Proof. Direct computations allow us to observe that f^ 1 ^ is given by the 
formula f^^^x) = u~ 1 (x)xu(x), x G fi. The second part may be proved 
similarly as in the previous example. □ 
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Remark 2. We would like to point out that the situation occurring in 
the above example may be described more precisely. Namely, put u := 



conjugation J u : x i— > u(x)xu(x) 1 is an automorphism of Q if and only if a 
depends on £ 12 , trx and detx. 

The problem of describing the group of automorphisms of spectral ball 
was divided by Rostand ([I]) into two parts. The first one is to prove that any 
normalized automorphism (i.e. an automorphism fixing the origin and satys- 
fying /'(0) = id) is a conjugation J u , where u : Q — > M.^ 1 is a holomorphic 
mapping. The second one is to show that the automorphism being a conju- 
gation is of the form J u with u satisfying the condition u(q~ 1 xq) = u(x) for 
each x e Q and each invertible q. Note that it was shown in this paper that 
the second statement is false. On the other hand it seems that the answer 
to the first question is affirmative. So we have two following problems: 

• Is every normalized automorphism of the spectral ball of the form 
x — > u(x)xu(x)~ 1 , where u : Q — )■ .M" 1 is a holomorphic mapping? 

• Describe all holomorphic u : f2 — > M.^ 1 such that the mapping 
x i — y m(x)x«(i) _1 is an automorphism of fl. 
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where a is an arbitrary holomorphic function on f2. Then the 
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